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DETAILED ACTION 

Response to Amendment 

1 . This communication is in response to the amendment filed on June 01 , 
2009. 

Claims 6-8, 11, 20, 33-34, 36, and 39 are canceled. 
Claims 1, 9-10, 18-19, 35, and 38 are amended. 
Claims 40-41 are newly added. 

Claims 1-5, 9-10, 12-19, 21-32, 35, 37-38, and 40-41 are pending. 

Response to Arguments 

2. Objection to claim 35 is withdrawn in view of Applicant's amendment. 

3. 35 U.S.C. 112, first and second paragraph, rejections of claims 18-19, 21- 
26, 32, and 38 are withdrawn in view of Applicant's amendment. 

4. Applicant's arguments with respect to the 35 U.S.C. 103(a) rejections of 
claims 1-5, 9-10, 12-19, 21-32, 35, and 37-38 have been considered but are 
moot in view of the new ground of rejections presented hereon. 

5. Rejections for newly added dependent claims 40-41 are also presented 
hereon. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

7. Claims 1-5, 9, 12-13, 17, 27-31, 35, 37, and 41 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Nal<amura et al. (Pub. No. US 

2004/0201740, filed on March 15, 2002; hereinafter Nakamura ) in view of 
Andrews et al. {Pub. No. US 2003/0060979, filed on September 21, 2001; 
hereinafter Andrews ), and further in view of Brechner et al. (Pat. No. US 
6,741,996, filed on April 18, 2001; hereinafter Brechner ). 

Regarding claim 1. Nakamura clearly shows and discloses a method 

([Abstract]) comprising: 

obtaining user provided information in consequence to any user operation 
on a mobile terminal device {Image capture device, such as a digital camera or a 
digital video camera 23, captures original images, [0021]); 

obtaining context information associated with said user provided 
information, wherein said context information is related to at least one current 
condition of the mobile terminal device at the time of said any user operation and 
includes calendar information from an electronic calendar implemented at said 
mobile terminal device {Image metadata is essentially non-picture information 
that is stored along with picture information in a file and can include information 
such as the date and time the picture was taken, whether a flash was used. 
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which camera model was used, camera settings such as zoom and exposure, 
location information such as GPS- derived data, and audio annotations, [0025]); 

obtaining a first time information in accordance witli said user provided 
information {In step S314 image metadata that is stored along with each image is 
extracted. Image metadata are preferably date events that identify a date and 
time for specific events, [0031]); 

obtaining one or more calendar entries includes in said calendar 
information, wherein each calendar entry comprises a second time information 
with a start time and an end time {Each calendar in turn contains one or more 
events, each of which are preferably identified by an event title. Examples of 
event titles include birthdays for specific family members like "Dad's birthday" or 
"Nicholas's Birthday, " vacation events that correspond to vacations at specific 
times of the yearlil<e "Summer break," and "Spring break", [0030]); 

comparing said first time information with each of said second time 
information to identify matching calendar entries by assigning a membership 
function to each of said second time information {date events stored along with 
image as metadata are compared with dates corresponding to events specified in 
the calendar Where an image meta data corresponds to more than one event on 
the calendar, and more than one meta data are available for the image, the other 
meta data are utilized to determine a storage location for that image, [0033]) and 
deriving a membership grade value from each of said membership functions in 
accordance with said first period of time {where two events are scheduled on the 
calendar for the same date, and the calendar schedules events by date and time. 
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the comparing step compares the time the image was recorded (meta data), if 
available, against the time of each of the calendar events on that date, and 
determines a location for the image accordingly, [0034]), wherein said 
membersliip function is a function in time, which rises from zero value at a 
predetermined moment in time before said start time of a respective calendar 
entry, and becomes zero value at a predefined moment in time after said end 
time of said respective calendar entry, wherein membership function defines an 
extended timeframe for each calendar entry {comparisons of taken pictures and 
calendar entries to determine a match with calendar entries, [0040]-[0051]); 

obtaining meta-information from each matching calendar entry of said one 
or more obtained calendar entries (As shown in step S318, an affirmative 
response to this inquiry results in the creation of an album whose title is the title 
of the calendar event corresponding to the match, [0036]). 

Nakamura does not disclose assigning meta-information obtained from 
said context information and said membership grade value to said user provided 
information, said membership grade value defining a measure which allows to 
estimate a reliability of a timely relatedness for retrieval; storing said user 
provided information and said meta-information in a history storage in order to 
establish an information history functionality, and wherein said meta-information 
is employable for retrieval of said user provided information by matching request 
information of a retrieval request with said meta-information for selecting a user 
provided information assigned to said meta-information matched to said request 
information. 
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However, Andrews discloses: 

assigning meta-information obtained from said context information and 
said memberslnip grade value to said user provided information {the data 
collection table 31 5C is a table organized as a series of columns and rows. Each 
row defines a record for a particular appointment. The columns include an 
appointment column 502A, an alarm lead time column 502B, a calendar event 
column 502C, a GPS departure location column 502D, a GPS arrival location 
502E, an arrival time column 502F and a stress factor column 502G. Each entry 
under the appointment column 502A contains an appointment number by which 
the particular appointment for that record can be identified, [0042]), said 
membership grade value defining a measure which allows to estimate a reliability 
of a timely relatedness for retrieval {stress factor in Figure 5); and 

storing said user provided information and said meta-information in a 
history storage in order to establish an information history functionality {Referring 
now to Figure 7, a summary view 31 5B of the history data is shown. In this view, 
the history data contained in the history data structure 315 has been sorted into 
two columns 702 and 704. Lead times and current GPS locations are contained 
in the history data structure among with other attributes, [0047]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Andrews with the 
teachings of Nakamura for the purpose of notifying a user of an appointment by 
estimating a travel time for the user to travel from a current location of the 
wireless device to a appointment location ([Abstract] of Andrews ). 
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Nakamura . as modified by Andrews , does not disclose wlierein said meta- 
information is employable for retrieval of said user provided information by 
matching request information of a retrieval request with said meta-information for 
selecting a user provided information assigned to said meta-information matched 
to said request information. 

However, Brechner discloses: 

wherein said meta-information is employable for retrieval of said user 
provided information by matching request information of a retrieval request with 
said meta-information for selecting a user provided information assigned to said 
meta-information matched to said request information {the user could also 
specifically identify digital pliotograplis relating to beacli scenes by searcliing for 
metadata that include the word "beach" within the meta data associated with 
each of the files included within the collection hierarchy stored in the user's 
content database, [Column 11, Lines 11-23]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Brechner with the 
teachings of Nakamura . as modified by Andrews , for the purpose of automating 
the indexing of media content in a simplified hierarchical storage scheme and the 
automated assignment of keywords and other metadata to the media content to 
facilitate its retrieval ([Column 1, Lines 5-10] of Brechner ). 
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Regarding claim 2, Brechner further discloses said assigning comprises: 

extracting said meta-information from said context information (as the 
content is being imported into database created by the Clip Organizer, it 
simultaneously indexed the contents by adding keywords or metadata based 
upon contextual information, [Column 6, Lines 6-11]). 

Regarding claim 3, Brechner further discloses said obtaining of said user 
provided information comprises: 

receiving user input information being generated by user operation of any 
input means of said mobile terminal device and/or 

receiving transaction information and/or communication information being 
received via any communication interface of said mobile terminal device 

{accessing those media files can be done with computing devices in distributed 
computing environments that include remote processing devices United through a 
communication network, [Column 4, Lines 29-32]), 

wherein said user input information and/or transaction information and/or 
said communication information represent said user provided information {media 

files generated from digital cameras, web site image files, home video editing 
files, sound files, and other types of user media content are stored on a hard 
drive, [Column 1, Lines 14-18]). 
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Regarding claim 4, Brechner further discloses said assigning of said 
meta-information to said user provided information comprises: 

extracting labeling information and/or indexing information from said 

context information (a user might store a digital photograph file "beachi jpg"that 
was taken with an OLYMPUS™ digital camera on the beach during a vacation to 
Hawaii in 1999 in a folder having a path such as "C:/My Documents/My 
Photo/Digital Camera/OlympusA/acations+HawaiH 999Beach1.jpg." This path 
would be parsed by the Clip Organizer software to identify keywords for inclusion 
in the metadata for the file. These keywords would include: "My Documents, " "My 
Photos, " "Digital Camera, " "Olympus, " "Vacations, " "Hawaii, " and "1999. ", 
[Column 10, Lines 47-57]); 

assigning said labeling information and/or indexing information to said 

user provided information (as the content is being imported into database created 
by the Clip Organizer, it simultaneously indexed the contents by adding keywords 
or metadata based upon contextual information, [Column 6, Lines 6-11]); and 

storing said labeling information and/or indexing information and said user 
provided information {content is being imported into a database created by the 
Clip Organizer, metadata based on contextual information is also added to 
enable a user to find the personal content at a later time using a keyword search, 
[Column 6, Lines 6-11]); 

wherein said labeling information and/or indexing information is employed 
as said meta-information for establishing retrieval functionality of said user input 
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history {The user could also specifically identify digital photographs relating to 
beach scenes by searching for metadata that include the word "beach" within the 
metadata associated with each of the files included within the collection hierarchy 
stored in the user's content database, [Column 1 1 , Lines 11 -23]). 

Regarding claim 5, Brecliner further discloses said associating said 
context information with said user provided information comprises: 

generating referencing information for at least a part of said user provided 
information {each collection in the database is associated a full path that 
indicates the location on the hard drive of the collection, [Column 9, Lines 53- 
55]); 

storing said referencing information in a first storage area {the original 
media file with full path address would serve as a reference in order to import 
that file into the database created by Clip Organizer, [Column 9, Lines 48-59]); 
and 

storing said part of said user provided information in a second storage 
area; wherein said referencing information comprises address information which 
allow to retrieve said part of user provided information from said second storage 
area {Provision of this full path is important, because it sets up an import and 
indexing component and a browse import component, [Column 9, Lines 55-65]). 
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Regarding claim 9. Andrews further discloses partitioning said user 
provided Information Into a plurality information parts, each Information part 
relates to a maximum pre-defined period of time, each Information part being 
matched separately {the first lead time may define an alarm time six iiours prior 
to the projected start time while a user is at home preparing for the appointment. 
A second lead time may be a lunchtime reminder with a lead time of two hours. A 
third lead time may be 15 minutes before the projected start time, [0031]). 

Regarding claim 12, Andrews further discloses each of said plurality of 
calendar entries being included in said calendar information is semantically 
structured and said obtaining of meta-information from each matching calendar 
entry {Figure 5-6) comprises: 

obtaining of meta-information from each of said matching semantically 

structured calendar entry {Once the alarm time is reached, processing continues 
to step 908 where the current GPS location and the appropriate calendar 
information (i.e., the user's current activity based on a calendar entry) are 
extracted, [0058]). 

Regarding claim 13, Brechner further discloses: 

obtaining said user provided information including user provided audio 
information and in parallel additional user provided information, said user 
provided audio Information being recorded and stored {media files generated 
from digital cameras, web site image files, home video editing files, sound files. 



Application/Control Number: 10/540,440 Page 12 

Art Unit: 2165 

and other types of user media content are stored on a hard drive, [Column 1 , 
Lines 14-18]); 

obtaining said context information in parallel to said user provided audio 
information, said context information comprising user input information generated 
in consequence to user action against said mobile terminal device (a user might 
store a digital photograph file "beach 1 jpg" that was taken with an OLYMPUS™ 
digital camera on the beach during a vacation to Hawaii in 1999 in a folder 
having a path such as "C:/My Documents/My Photo/Digital 
Camera/OlympusA/acations+HawaiH 999Beach1.jpg." This path would be parsed 
by the Clip Organizer software to identify keywords for inclusion in the metadata 
for the file. These keywords would include: "My Documents, " "My Photos, " 
"Digital Camera," "Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, 
Lines 47-57]); and 

generating meta-information comprising information about said additional 
user provided information and said context information (as the content is being 
imported into database created by the Clip Organizer, it simultaneously indexed 
the contents by adding keywords or metadata based upon contextual 
information, [Column 6, Lines 6-11]). 

Regarding claim 17, Brechner further discloses said meta-information is 
displayed by assigning graphical elements to each information entry included in 
said meta-information and predicting said graphical elements illustrating the 
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content of the meta-information and showing associations defined in said meta- 
information (Figure 10). 

Regarding claim 27, Brechner further discloses a method for retrieving 
user provided information being organized, comprising: 

receiving a request for retrieving, said request comprising request 
information for instructing to retrieve certain user provided information [If the user 
wants to search for keywords in the metadata associated with the media files, the 
user will enter the likely keywords in a text block 354, [Column 1 5, Lines 37-44]); 

comparing said request information with said meta-information being 
assigned to said user provided information which is provided by said information 
history functionality {In this example, the user has entered the text "Vacation," 
indicating that any media file having that text string in its path and thus 
automatically included in the metadata for the file as a result of the automatic 
indexing of the media files will be located by the search, [Column 15, Lines 37- 
44]); 

retrieving said user provided information being assigned to said meta- 
information which matches said request information {any media file that has the 
searched word "Vacation" in its metadata will be located by the search, [Column 
15, Lines 37-44]); 
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generating a response comprising said retrieved user provided information 

{When the Search control is selected, a list of all media files meeting the desired 
criteria will be displayed to the user, [Colum 15, Lines 53-55]); and 

transmitting said response {the search results are displayed on the 
monitor, [Column 15, Lines 53-55] and Figure 1). 

Regarding claim 28, Brecliner further discloses said retrieving of said user 
provided information comprises: 

retrieving referencing information being associated with said user provided 
information to be retrieved, said referencing information comprising address 
information which addresses said part of user provided information being stored 
in a second storage area (a user might store a digital photograph file "beachi 
jpg" that was taken with an OLYMPUS™ digital camera on the beach during a 
vacation to Hawaii in 1999 in a folder having a path such as "C:/My 
Documents/My Photo/Digital 

Camera/OlympusA/acations+HawaiH 999Beach1.jpg." This path would be parsed 
by the Clip Organizer software to identify keywords for inclusion in the metadata 
for the file. These keywords would include: "My Documents, " "My Photos, " 
"Digital Camera," "Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, 
Lines 47-57]); and 

retrieving said part of user provided information from said second storage 
area {The user could also specifically identify digital photographs relating to 
beach scenes by searching for metadata that include the word "beach" within the 
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metadata associated with each of the files included within the collection hierarchy 
stored in the user's content database, [Column 1 1 , Lines 1 1-23]). 

Regarding claim 29. Andrews furtlier discloses, wherein said retrieving of 
said user provided information comprises: 

evaluating said user provided information being retrieved on the basis of 
said membership grade values obtained from calendar entries, said membership 
grade value defining a measure which allows to estimate a reliability for retrieval 
{The particular appointment is identified in the appointment column 602A. The 
entries of the lead time column 602B are populated using the alarm lead time 
column 502B of the history data structure 315. Each entry of a projected lead 
time column 602C provides a lead time to be used for the next appointment. The 
projected lead time is calculated using a time difference value contained in the 
entries of a time difference column 602D, [0044]). 

Regarding claim 30, Brechner clearly shows and discloses a method for 

providing storage capacity for organizing user provided information being 
provided with meta-information {Abstract), comprising: 

receiving a request to store at least a part of said user provided 
information {Figure 11 shows an interface to organize selected media clips on 
command. This interface could be displayed on hand-held devices, pocket 
personal computing devices, [Column 4, Lines 23-29]), wherein said request 
comprises said part of said user provided information and referencing information 
{Figure 11, #324 & #326); 
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storing said part of said user provided information and said referencing 

information such that said part of said user provided information is retrievable In 
conjunction with said referencing information (a user might store a digital 
photograpli file "beaclil jpg" that was taken with an OLYMPUS™ digital camera 
on the beach during a vacation to Hawaii in 1999 in a folder having a path such 
as "CMy Documents/My Photo/Digital Camera / Olympus / 
Vacations+HawaiH 999Beach1.jpg." This path would be parsed by the Clip 
Organizer software to identify keywords for inclusion in the metadata for the file. 
These keywords would include: "My Documents, " "My Photos, " "Digital Camera, " 
"Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, Lines 47-57]). 

receiving a request for retrieving at least said part of said user provided 
Information wherein said request comprises referencing information {If the user 
wants to search for keywords in the metadata associated with the media files, the 
user will enter the likely keywords in a text block 354, [Column 1 5, Lines 37-44]); 

retrieving said part of said user provided information in accordance with 
said referencing information {In this example, the user has entered the text 
"Vacation, " indicating that any media file having that text string in its path and 

thus automatically included in the metadata for the file as a result of the 
automatic indexing of the media files will be located by the search, [Column 1 5, 
Lines 37-44]); and 
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generating a response including said retrieved part of said user provided 

information {When the Search control is selected, a list of all media files meeting 
the desired criteria will be displayed to the user, [Colum 15, Lines 53-55]). 

Regarding claim 31, Brechner clearly shows and discloses a computer 
readable storage medium embedded with a computer program comprising 
programming code for carrying out the operations of claim 1 ([Column 4, Lines 
12-17]). 

Regarding claim 35, Nakamura clearly shows and discloses an apparatus 
{Figure 1) comprising: 

a component for obtaining user provided information in consequence to 
any user operation on a mobile terminal device {Image capture device, such as a 
digital camera or a digital video camera 23, captures original images, [0021]); 

a component for obtaining context information associated with said user 
provided information, wherein said context information is related to at least one 
current condition of the mobile terminal device at the time of said any user 
operation and includes calendar information from an electronic calendar 
implemented at said mobile terminal device {Image metadata is essentially non- 
picture information that is stored along with picture infomnation in a file and can 
include information such as the date and time the picture was taken, whether a 
flash was used, which camera model was used, camera settings such as zoom 
and exposure, location information such as GPS- derived data, and audio 
annotations, [0025]); 
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a component for obtaining a first time information in accordance witli said 
user provided information {In step S314 image metadata that is stored along with 
each image is extracted. Image metadata are preferably date events that identify 
a date and time for specific events, [0031]); 

a component for obtaining one or more calendar entries includes in said 
calendar information, wherein each calendar entry comprises a second time 
information with a start time and an end time {Each calendar in turn contains one 
or more events, each of which are preferably identified by an event title. 
Examples of event titles include birthdays for specific family members like "Dad's 
birthday" or "Nicholas's Birthday, " vacation events that correspond to vacations at 
specific times of the year like "Summer break," and "Spring break", [0030]); 

a component for comparing said first time information with each of said 
second time information to identify matching calendar entries by assigning a 
membership function to each of said second time information {date events stored 
along with image as metadata are compared with dates corresponding to events 
specified in the calendar. Where an image meta data corresponds to more than 
one event on the calendar, and more than one meta data are available for the 
image, the other meta data are utilized to determine a storage location for that 
image, [0033]) and deriving a membership grade value from each of said 
membership functions in accordance with said first period of time {where two 
events are scheduled on the calendar for the same date, and the calendar 
schedules events by date and time, the comparing step compares the time the 
image was recorded (meta data), if available, against the time of each of the 
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calendar events on that date, and determines a location for tlie image 
accordingly, [0034]), wherein said membership function is a function in time, 
which rises from zero value at a predetermined moment in time before said start 
time of a respective calendar entry, and becomes zero value at a predefined 
moment in time after said end time of said respective calendar entry, wherein 
membership function defines an extended timeframe for each calendar entry 
{comparisons of taken pictures and calendar entries to determine a match with 
calendar entries, [0040]-[0051]); 

a component for obtaining meta-information from each matching calendar 
entry of said one or more obtained calendar entries {As shown in step S318, an 
affirmative response to this inquiry results in the creation of an album whose title 
is the title of the calendar event corresponding to the match, [0036]). 

Nakamura does not disclose assigning meta-information obtained from 
said context information and said membership grade value to said user provided 
information, said membership grade value defining a measure which allows to 
estimate a reliability of a timely relatedness for retrieval; storing said user 
provided information and said meta-information in a history storage in order to 
establish an information history functionality, and wherein said meta-information 
is employable for retrieval of said user provided information by matching request 
information of a retrieval request with said meta-information for selecting a user 
provided information assigned to said meta-information matched to said request 
information. 

However, Andrews discloses: 



Application/Control Number: 10/540,440 Page 
Art Unit: 2165 

assigning meta-information obtained from said context information and 
said membership grade value to said user provided information {the data 
collection table 31 5C is a table organized as a series of columns and rows. Each 
row defines a record for a particular appointment. The columns include an 
appointment column 502A, an alarm lead time column 502B, a calendar event 
column 502C, a GPS departure location column 502D, a GPS arrival location 
502E, an arrival time column 502F and a stress factor column 502G. Each entry 
under the appointment column 502A contains an appointment number by which 
the particular appointment for that record can be identified, [0042]), said 
membership grade value defining a measure which allows to estimate a reliability 
of a timely relatedness for retrieval (stress factor in Figure 5); and 

storing said user provided information and said meta-information in a 
history storage in order to establish an information history functionality {Referring 
now to Figure 7, a summary view 31 5B of the history data is shown. In this view, 
the history data contained in the history data structure 315 has been sorted into 
two columns 702 and 704. Lead times and current GPS locations are contained 
in the history data structure among with other attributes, [0047]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Andrews with the 
teachings of Nakamura for the purpose of notifying a user of an appointment by 
estimating a travel time for the user to travel from a current location of the 
wireless device to a appointment location ([Abstract] of Andrews). 
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Nakamura . as modified by Andrews , does not disclose wlierein said meta- 
information is employable for retrieval of said user provided information by 
matching request information of a retrieval request with said meta-information for 
selecting a user provided information assigned to said meta-information matched 
to said request information. 

However, Brechner discloses: 

wherein said meta-information is employable for retrieval of said user 
provided information by matching request information of a retrieval request with 
said meta-information for selecting a user provided information assigned to said 
meta-information matched to said request information {the user could also 
specifically identify digital pliotograplis relating to beacli scenes by searcliing for 
metadata that include the word "beach" within the meta data associated with 
each of the files included within the collection hierarchy stored in the user's 
content database, [Column 11, Lines 11-23]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Brechner with the 
teachings of Nakamura . as modified by Andrews , for the purpose of automating 
the indexing of media content in a simplified hierarchical storage scheme and the 
automated assignment of keywords and other metadata to the media content to 
facilitate its retrieval ([Column 1, Lines 5-10] of Brechner ). 
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Regarding claim 37, Brechner further discloses: 

a component {Figure 1) for obtaining user provided audio information and 
additional user provided information (a user might store a digital photograph file 
"beachi jpg" that was taken with an OLYMPUS™ digital camera on the beach 
during a vacation to Hawaii in 1999 in a folder having a path such as "C:/My 
Documents/My Photo/Digital 

Camera/OlympusA/acations+HawaiH 999Beach1.jpg." This path would be parsed 
by the Clip Organizer software to identify keywords for inclusion in the metadata 
for the file. These keywords would include: "My Documents, " "My Photos, " 
"Digital Camera," "Olympus," "Vacations," "Hawaii," and "1999.", [Column 10, 
Lines 47-57]); 

a component {Figure 1) for recording said user provided audio information 

{before the sound/media files were stored on a computer's hard drive, it is 
inherent that the files had been recorded/downloaded/transferred from other 
sources, [Column 1, Lines 13-16]); 

a storage component for storing said user provided audio information 
{hard drive, [Column 1, Lines 13-16]); 

a component {Figure 1) for obtaining context information in parallel to said 
user provided audio information, said context information comprising user input 

information generated in consequence on a user action against said mobile 
terminal device (a user might store a digital photograph file "beachi jpg" that was 
taken with an OLYMPUS™ digital camera on the beach during a vacation to 
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Hawaii in 1999 in a folder tiaving a patti sucti as "C:/My Documents/My 
Plioto/Digital Camera/OlympusA/acations+HawaiH 999Beacli1.jpg. " This patli 
would be parsed by the Clip Organizer software to identify keywords for inclusion 
in the metadata for the file. These keywords would include: "My Documents, " "My 
Photos, " "Digital Camera, " "Olympus, " "Vacations, " "Hawaii, " and "1999. ", 
[Column 10, Lines 47-57]); and 

a component {Figure 1) for generating meta-information comprising 
information about said additional user provided information and said context 
information (as the content is being imported into database created by the Clip 
Organizer, it simultaneously indexed the contents by adding keywords or 
metadata based upon contextual information, [Column 6, Lines 6-11]). 

Regarding claim 41 . Nakamura further discloses said membership grade 

value indicates said timely relatedness between said user provided information 
and one of said matching calendar entries {where two events are scheduled on 
the calendar for the same date, and the calendar schedules events by date and 
time, the comparing step compares the time the image was recorded (meta 
data), if available, against the time of each of the calendar events on that date, 
and determines a location for the image accordingly, [0034]) 
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8. Claims 10, and 15-16 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Nakamura et al. {Pub. No. US 2004/0201740, filed on March 
15, 2002; hereinafter Nakamura ) in view of Andrews et al. {Pub. No. US 
2003/0060979, filed on September 21, 2001; hereinafter Andrews ), in view of 
Brechner et al. {Pat. No. US 6,741,996, filed on April 18, 2001; hereinafter 
Brechner), and further in view of Gupta et al. (Pat. No. US 6,484,156, filed on 
September 15, 1999; hereinafter Gupta ). 

Regarding claim 10, Nakamura . as modified by Andrews and Brechner . 
does not disclose in case said first period of time exceeds said second period of 
time: sectioning said user provided information into at least two information 
sections, one of said at least two information sections fitting with said second 
period of time, said one fitting information section being matched. 

However, Gupta discloses in case said first period of time exceeds said 
second period of time {Figure 10 shows a target file is being divided into multiple 
segments, each segment has a corresponding play time, [Column 8, Lines 10- 
23]): 

sectioning said user provided information into at least two information 
sections, one of said at least two information sections fitting with said second 
period of time, said one fitting information section being matched {Figure 10 
shows if the search time from 000:04:08.262 to 000:11:00.14 matches a 
segment, that segment will be retrieved when the 'Play' button is pressed. See 
further Figure 4 for all searchable fields in an annotation entry). 
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It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of Gupta with the 
teachings of Nakamura, as modified by Andrews and Brechner . for the purpose 
of providing easy access to different multimedia streams by using a group 
identifier to identify all of the multimedia streams and their corresponding 
annotations ([Column 2, Lines 34-46] of Gupta ). 

Regarding claim 15. Gupta further discloses a recording context 
information at least including: 

information and time information about said user provided audio 
information {Figure 4, see Time Range #184); 

time information about said user input information Figure 4, see Creation 
Time #188); and 

information about said additional user provided information being 
associated with the user provided audio information {Figure 5 sliows tlie 
annotation entry used to associate witli tlie media). 

Regarding claim 16. Gupta further discloses said generating of said 
recording context information comprises: 

encoding said meta-information as a document being encoded in 
accordance with a markup language {communication between client 15 and 
se/ver 10 is performed via HTTP, using commands encoded as Uniform 
Resource Locators (URLs) and data formatted as object linking and embedding 
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(OLE) structured storage documents, or alternatively using Extensible Markup 
Language (XML), [Column 6, Lines 9-15]). 

9. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. {Pub. No. US 2004/0201740, filed on Marcti 15, 2002; 
tiereinafter Nakamura ) in view of Andrews et al. (Pub. No. US 2003/0060979, 
filed on September 21, 2001; hereinafter Andrews ) . and further in view of 
Brechner et al. {Pat. No. US 6,741,996, filed on April 18, 2001; tiereinafter 
Brechner), and further in view of Asazu (Pub. No. 2001/0049691, published on 
December 6, 2001). 

Regarding claim 14, Nakamura . as modified bv Andrews and Brechner . 
further discloses said user input information comprises control signals related to 
said recording of said user provided audio information, further comprises: 

on receiving a keyword signal: initiating a recording of a user provided 
audio keyword information including keywords relating to said user provided 
audio information {the file suffix is added as a keyword in the metadata for the file 
in a block 180. The suffix is the media file extension, such as "wav" for audio files 
in the wave format, [Column 10, Lines 32-44] of Brechner); 

Asazu then discloses: 

on receiving a start signal: initiating said recording and storing of said user 

provided audio information {The following processing is executed in response to 
the query #2. Firstly, the QT plug-in component 14 calls a function of start () to 
the recorder 25 to issue an instruction to start recording of media data. 
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Subsequently, the recorder 25 notifies ttie logger 24 of the instruction to start 
recording of media data, [0068]); 

on receiving an attach signal: associating additional information with said 
user provided audio information {In this case, the recorder 25 calls a function of 
notify ()to the recording manager 26 whenever each frame is processed. The 
recording manager 26 activates the DA plug-in component 15 registered by the 
system in advance per frame data. Each DA plug-in component 15 executes an 
analysis of frame data to create meta-data, [0068]); 

on receiving a pause signal: pausing said recording and storing of said 
user provided audio information {The following processing is executed in 
response to the query #3. The QT plug-in component 14 calls a function of stop ( 
) to the recorder 25 to issue an instruction to stop recording of media data. The 
recorder 25 stops recording of media data, while notifying the logger 25 that 
recording of media data is stopped, [0069]); 

on receiving a continue signal: resuming said recording and storing of said 
user provided audio information subsequently to said pausing of said recording 
and storing {the processing for acquiring the information relating to meta-data is 
interrupted for the duration of the above access until the end of the other 
transaction. Thereafter, the QT plug-in component 14 resumes the processing to 
acquire the information relating to the required meta-data, [0078]. It is also very 
well-known in the art that resuming a paused recording part would start at the 
pausing point like in a digital voice recorder); and 
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on receiving a stop signal: stopping said recording and storing of said user 
provided audio information and generating said recording context information 
{The QT plug-in component 14 calls a function of commit () to the recorder 25, 
resulting in completion of recording of media data. It is to be understood that 
calling a function of abort ( ) instead of the function of commit ( ) aborts all the 
media data and meta-data recorded or created in this session after calling of the 
function of start ( ), and the system is restored to its original state, [0069]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of Asazu with the 
teachings of Nakamura . as modified by Andrews and Brechner . for the purpose 
of providing media data management functions including replaying, readout, 
recording, deletion and synthesis of media data by the use of meta-data ([0013] 
and Abstract of Asazu ). 

10. Claims 18, 23, 32, and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable Andrews et al. {Pub. No. US 2003/0060979, filed on September 21, 
2001; hereinafter Andrews ) in view of Brechner et al. {Pat. No. US 6,741,996, 
filed on April 18, 2001; hereinafter Brechner ), further in view of LaDue {Pub. No. 
US 2003/0133423, filed on November 15, 2002) and further in view of Tecu et al. 
{Pub. No. US 2004/0034655, filed on July 17, 2002; hereinafter Tecu). 
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Regarding claim 18, Andrews clearly shows and discloses a method 
([Abstract]) comprising: 

obtaining user provided information in consequence to any user operation 

on a mobile terminal device {Image capture device, such as a digital camera or a 
digital video camera 23, captures original images, [0021]); 

obtaining context information associated with said user provided 
information, wherein said context information is related to at least one current 
condition of the mobile terminal device at the time of said any user operation 
{Image metadata is essentially non-picture information that is stored along with 
picture information in a file and can include information such as the date and time 
the picture was taken, whether a flash was used, which camera model was used, 
camera settings such as zoom and exposure, location information such as GPS- 
derived data, and audio annotations, [0025]); and 

storing said user provided information with coded meta-information in a 
history storage in order to establish an information history functionality {Referring 
now to Figure 7, a summary view 31 5B of the history data is shown. In this view, 
the history data contained in the history data structure 315 has been sorted into 
two columns 702 and 704. Lead times and current GPS locations are contained 
in the history data structure among with other attributes, [0047]). 

Andrews does not explicitly disclose wherein said meta-information is 
employable for retrieval of said user provided information by matching request 
information of a retrieval request with said meta-information for selecting a user 
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provided information assigned to said meta-information matched to said request 
information. 

However, Brechner discloses: 

wlierein said meta-information is employable for retrieval of said user 
provided information by matching request information of a retrieval request with 
said meta-information for selecting a user provided information assigned to said 
meta-information matched to said request information {the user could also 
specifically identify digital photographs relating to beach scenes by searching for 
metadata that include the word "beach" within the meta data associated with 
each of the files included within the collection hierarchy stored in the user's 
content database, [Column 11, Lines 11-23]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Brechner with the 
teachings of Andrews for the purpose of automating the indexing of media 
content in a simplified hierarchical storage scheme and the automated 
assignment of keywords and other metadata to the media content to facilitate its 
retrieval ([Column 1, Lines 5-10] of Brechner ). 

LaDue then discloses: 

providing code basis representing a plurality of coding symbols, said code 
basis comprising a pre-defined number of pre-defined frequencies, wherein a 
plurality of coding symbols represents a character and symbol code table 
employable for coding said meta-information {Figure 3 depicts a set of 
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fundamental semantic constructs specific to OPD, 76, tlieory and practice. For 
example, octave pulse values can be expressed in numeric characters arranged 
in an absolute progression, 77. Each numeric character, 83, has a corresponding 
harmonic octave value attribute, 84, [01 15]), wherein said set of code basis is 
defined within a first frequency range, which is one frequency range of a plurality 
of frequency ranges forming a total frequency range being applicable to said user 
provided audio information {Figure 3 depicts the basic conceptual constructs of 
octave pulse data, 76. The musical harmonic value, 81a-b and 84, of each 
depicted octave pulse construct and each corresponding alpha numeric 
character, 77, 78 and 79, are all depicted, [0221]); 

repeating said code basis within at least one further frequency range out 
of said plurality of frequency ranges {Octave pulses that are transported within 
PCM signal constructs must contend with noisy switch exchanges and E1/T1 
repeaters while traveling through selected PLMN and PSTN networks. The 
invention creates a "hanmonic paradox", as discussed in Figure 18, [0144]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of LaDue with the 
teachings of Andrews , as modified by Brechner , for the purpose of sending data 
communications over wireless digital voice communications networks which 
transmit voice communications in voice frames, each of which contains a 
digitized segment of a voice communication in a voice frame format ([Abstract] of 
LaDue ). 
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Tecu then discloses: 

coding said meta-information in accordance witli said code basis defined 
within said first frequency range and repeated within said at least one further 

frequency range to obtain redundancy {At step 116, encoder routine 26 
designates metadata 40 to be encoded at each of the selected frequencies 62. 
For example, each type of metadata 40 to be included in the particular data 
stream 32 may be encoded at each of a plurality of designated frequencies 62, 
[0028]. Encoder routine 26 may encode metadata 40 by generating a bit pattern 
at one or more desired inaudible frequencies 62, [0022]); and 

combining said user provided audio information and said coded meta- 
information by embedding said coded meta-information into said user provided 
audio information {metadata 40 may be encoded at a frequency 62 of 
approximately 20 kHz or greater, thereby rendering the encoded metadata 40 
inaudible to human hearing. If data stream 32 is to be compressed, the encoded 
metadata 40 may be inserted into audio data 34 either before or after 
compression, thereby providing additional functionality and versatility to system 
10, [00\19]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of LaDue with the 
teachings of Andrews , as modified by Brechner and Tecu , for the purpose of 
encoding metadata at a plurality of predetermined intensity levels at a human- 
inaudible frequency and populating the audio data of the data stream with the 
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encoded metadata using an encoder routine accessible by tine processor ([0003] 
of Tecu ). 

Regarding claim 23, Tecu furtlier discloses a method for extracting meta- 
information from an audio information having embedded said meta-information, 
said method comprising: 

providing a correlation basis comprising each frequency being included in 
a set of code bases representing a plurality of coding symbols {Figure 1, 
frequency data 60), said coding symbols being employed for coding said meta- 
information {relational data 50 may be generated after decoding of metadata 40 
by decoder routine 28, or relational data 50 may be generated upon encoding or 
insertion of metadata 40 into a particular data stream 32, [0021]); and 

applying said correlation basis onto said user provided audio information 
having embedded said meta-information to extract said meta-information 
{Processor 16 also generates relational data 50 corresponding to the encoded 
metadata 40 sucli tliat metadata 40 may be correlated to particular data streams 
32, [0021]); 

wherein said extracted meta-information being available for retrieval 
{searcli engine 20 may be used to quickly and efficiently locate a particular data 
stream 32 using searcli parameters corresponding to metadata 4, [0021]). 
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Regarding claim 32, Andrews clearly shows and discloses a computer 
readable storage medium embedded with a computer program comprising 
programming code for carrying out the operations of claim 18 {Figure 3). 

Regarding claim 38, Andrews clearly shows and discloses an apparatus 

{Figure 3) comprising: 

a component for obtaining user provided information in consequence to 
any user operation on a mobile terminal device {Image capture device, sucli as a 
digital camera or a digital video camera 23, captures original images, [0021]); 

a component for obtaining context information associated with said user 
provided information, wherein said context information is related to at least one 
current condition of the mobile terminal device at the time of said any user 
operation {Image metadata is essentially non-picture information that is stored 
along with picture information in a file and can include information such as the 
date and time the picture was taken, whether a flash was used, which camera 
model was used, camera settings such as zoom and exposure, location 
information such as GPS- derived data, and audio annotations, [0025]); and 

a storage component for storing said user provided information with coded 
meta-information in a history storage in order to establish an information history 
functionality {Referring now to Figure 7, a summary view 31 5B of the history data 
is shown. In this view, the history data contained in the history data structure 315 
has been sorted into two columns 702 and 704. Lead times and current GPS 
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locations are contained in ttie history data structure among with other attributes, 
[0047]). 

Andrews does not explicitly disclose wherein said meta-infornnation is 
employable for retrieval of said user provided information by matching request 
information of a retrieval request with said meta-information for selecting a user 
provided information assigned to said meta-information matched to said request 
information. 

However, Brechner discloses: 

wherein said meta-information is employable for retrieval of said user 
provided information by matching request information of a retrieval request with 
said meta-information for selecting a user provided information assigned to said 
meta-information matched to said request information {the user could also 
specifically identify digital photographs relating to beach scenes by searching for 
metadata that include the word "beach" within the meta data associated with 
each of the files included within the collection hierarchy stored in the user's 
content database, [Column 11, Lines 11-23]). 

It would have been obvious to an ordinary person skilled in the art at the 
time of the invention was made to incorporate the teachings of Brechner with the 
teachings of Andrews for the purpose of automating the indexing of media 
content in a simplified hierarchical storage scheme and the automated 
assignment of keywords and other metadata to the media content to facilitate its 
retrieval ([Column 1, Lines 5-10] of Brechner ). 



Application/Control Number: 10/540,440 
Art Unit: 2165 



Page 



LaDue then discloses: 

providing code basis representing a plurality of coding symbols, said code 
basis comprising a pre-defined number of pre-defined frequencies, wherein a 
plurality of coding symbols represents a character and symbol code table 
employable for coding said meta-information {Figure 3 depicts a set of 
fundamental semantic constmcts specific to OPD, 76, tlieory and practice. For 
example, octave pulse values can be expressed in numeric characters arranged 
in an absolute progression, 77. Each numeric character, 83, has a corresponding 
harmonic octave value attribute, 84, [01 15]), wherein said set of code basis is 
defined within a first frequency range, which is one frequency range of a plurality 
of frequency ranges forming a total frequency range being applicable to said user 
provided audio information {Figure 3 depicts the basic conceptual constructs of 
octave pulse data, 76. The musical harmonic value, 81a-b and 84, of each 
depicted octave pulse construct and each corresponding alpha numeric 
character, 77, 78 and 79, are all depicted, [0221]); 

repeating said code basis within at least one further frequency range out 
of said plurality of frequency ranges {Octave pulses that are transported within 
PCM signal constructs must contend with noisy switch exchanges and E1/T1 
repeaters while traveling through selected PLMN and PSTN networks. The 
invention creates a "hanmonic paradox", as discussed in Figure 18, [0144]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of LaDue with the 
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teachings of Andrews , as modified by Brechner . for the purpose of sending data 

communications over wireless digital voice communications networks which 
transmit voice communications in voice frames, each of which contains a 
digitized segment of a voice communication in a voice frame format ([Abstract] of 
LaDue). 

Tecu then discloses: 

coding said meta-information in accordance with said code basis defined 
within said first frequency range and repeated within said at least one further 
frequency range to obtain redundancy {At step 116, encoder routine 26 
designates metadata 40 to be encoded at each of the selected frequencies 62. 
For example, each type of metadata 40 to be included in the particular data 
stream 32 may be encoded at each of a plurality of designated frequencies 62, 
[0028]. Encoder routine 26 may encode metadata 40 by generating a bit pattern 
at one or more desired inaudible frequencies 62, [0022]); and 

combining said user provided audio information and said coded meta- 
information by embedding said coded meta-information into said user provided 
audio information {metadata 40 may be encoded at a frequency 62 of 
approximately 20 kHz or greater, thereby rendering the encoded metadata 40 
inaudible to human hearing. If data stream 32 is to be compressed, the encoded 
metadata 40 may be inserted into audio data 34 either before or after 
compression, thereby providing additional functionality and versatility to system 
10, [00\19]). 
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It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of LaDue with the 
teachings of Andrews , as modified by Brechner and Tecu . for the purpose of 
encoding metadata at a plurality of predetermined intensity levels at a human- 
inaudible frequency and populating the audio data of the data stream with the 
encoded metadata using an encoder routine accessible by the processor ([0003] 
of Tecu ). 

1 1 . Claims 19, 21-22, 24, and 26 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Andrews et al. (Pub. No. US 2003/0060979, filed on 
September 21 , 2001; hereinafter Andrews ) in view of Brechner et al. IPat. No. 
US 6,741,996, filed on April 18, 2001; hereinafter Brechner ). further in view of 
LaDue {Pub. No. US 2003/0133423, filed on November 15, 2002) and further in 
view of Tecu etal. (Pub. No. US 2004/0034655, filed on July 17, 2002; 
hereinafter Tecu ), and further in view of Tsuruoka {Pat. No. US 6,192,056, 
published on February 20, 2001). 

Regarding claim 19, Tecu discloses said combining comprises: 

obtaining a modulation signal from said coded meta-information {intensity 
data 70 having information associated with encoded metadata 40. For example, 
in the illustrated embodiment, intensity data 70 comprises signal amplitude or 
intensity levels 72 used to encode metadata 40 such that various intensity levels 
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72 may be used to designate a particular bit pattern of information, [001 7] of 
Tecu ). 

Andrews , as modified by Brecliner . LaDue and Tecu . does not disclose 
the other limitations. 

However, Tsuruoka discloses: 

obtaining a modulated signal by combining said user provided audio 
information and said modulation signal in a frequency domain (a modulated 
signal using a multiplicity of carriers wtiose frequency components are in an 
orthogonal relationsliip witli one anotlier, encodes data sucli as audio data or tlie 
like, and ttie encoded data are allocated to each carrier, thereby modulating each 
carrier, a digital signal in the frequency domain comprised of each modulated 
earner is inverse fast Fourier transformed into a digital signal in a time domain, 
[Column 1, Lines 23-31]); and 

obtaining said user provided audio information having embedded said 
meta-information by combining said modulated signal with said user provided 

audio information in a time domain (On its demodulating side, byA/D converting 
such an OFDM modulated signal and then applying the fast Fourier transform to 
the A/D converted signal, the encoded data allocated to each carrier is obtained, 
[Column 1, Lines 28-31]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of Tsuruoka with the 
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teachings of Andrews , as modified by Brechner . LaDue . and Tecu . for tine 

purpose of demodulating a digital ortliogonal frequency division multiplex 
modulated signal in which an information signal modulates a plurality of carriers 
whose frequency components are in an orthogonal relationship with one another 
using OFDM (Orthogonal Frequency Division Multiplex) modulation ([Column 1 , 
Lines 14-22] of Tsuruoka). 

Regarding claim 21 . Tsuruoka further discloses said obtaining of said 
modulated signal comprises: 

obtaining said modulated signal by multiplying said user provided audio 
information and said modulation signal in said frequency domain (a modulated 
signal using a multiplicity of carriers whose frequency components are in an 
orthogonal relationship with one another, encodes data such as audio data or the 
like, and the encoded data are allocated to each carrier, thereby modulating each 
earner, a digital signal in the frequency domain comprised of each modulated 
carrier is inverse fast Fourier transformed into a digital signal in a time domain, 
[Column 1, Lines 23-31]). 

Regarding claim 22, Tsuruoka further discloses: 

obtaining said user provided audio information having embedded said 
meta-information by adding said modulation signal and said user provided audio 

information in said time domain {On its demodulating side, byA/D converting 
such an OFDM modulated signal and then applying the fast Fourier transform to 
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the A/D converted signal, the encoded data allocated to each earner is obtained, 
[Column 1, Lines 28-31]). 

Regarding claim 24, Andrews , as modified by Tecu and Oxford , discloses 
said applying comprises: 

obtaining a modulation signal from said correlation basis {intensity data 70 
having information associated with encoded metadata 40. For example, in the 
illustrated embodiment, intensity data 70 comprises signal amplitude or intensity 
levels 72 used to encode metadata 40 such that various intensity levels 72 may 
be used to designate a particular bit pattern of information, [0017] of Tecu ): 

extracting magnitude signal values from said correlation signal {intensity 
data 70 comprises signal amplitude or intensity levels 72 used to encode 
metadata 40 such that various intensity levels 72 may be used to designate a 
particular bit pattem of information, [0017] of Tecu ). said magnitude signal values 
corresponding to each code basis of said set of code bases {encode metadata at 
a plurality of predetermined intensity levels at a human-inaudible frequency and 
populate the audio data of the data stream with the encoded metadata, [0003] of 
Tecu ): and 

evaluating said magnitude signal values to retrieve said meta-information 
from said user provided audio information {various intensity ranges 74 may also 

be used to designate a particular bit pattern of information. For example, a 
particular range of signal level strengths may be used to identify a bit designation 
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of "1 " while another range of signal level strengths may be used to identify a bit 
designation of'O", [0017] of Tecu ). 

Tsuruoka discloses: 

obtaining a correlation signal from said user provided audio information by 
convoluting said modulation signal and said user provided audio information 
having embedded said meta-information in a frequency domain (a modulated 
signal using a multiplicity of carriers whose frequency components are in an 
orthogonal relationship with one another, encodes data such as audio data or the 
like, and the encoded data are allocated to each carrier, thereby modulating each 
cannier, a digital signal in the frequency domain comprised of each modulated 
carrier is inverse fast Fourier transformed into a digital signal in a time domain, 
[Column 1, Lines 23-31]). 

Regarding claim 26, Andrews , as modified by Tecu and Oxford , discloses 
said correlation basis being defined in a first frequency range {frequency data 60 

in Figure 1, range is 20kHz or greater to render inaudible to human hearing, 
[0019] of Tecu ) and said obtaining of said correlation signal comprises : 

providing a correlation basis comprising all frequencies of said set of code 
bases within a first frequency range of a plurality of frequency ranges {frequency 
data 60 in Figure 1, range is 20kHz or greater to render inaudible to human 
hearing, [0019] of Tecu) : 
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mapping said correlation basis into eacli frequency range of a plurality of 
frequency ranges, said plurality of frequency ranges forming a total frequency 
range being applicable to said user provided audio information to obtain said 
modulation signal {Processor 16 also generates relational data 50 corresponding 
to the encoded metadata 40 such that metadata 40 may be correlated to 
particular data streams 32, [0021] of Tecu ): and 

Tsuruoka discloses: 

obtaining said correlation signal from said user provided audio information 
by convoluting said modulation signal and said user provided audio information in 
said frequency domain (a modulated signal using a multiplicity of carriers whose 
frequency components are in an orthogonal relationship with one another, 
encodes data such as audio data or the like, and the encoded data are allocated 
to each carrier, thereby modulating each carrier, a digital signal in the frequency 
domain comprised of each modulated carrier is inverse fast Fourier transformed 
into a digital signal in a time domain, [Column 1 , Lines 23-31]). 

1 2. Claim 25 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Andrews et al. {Pub. No. US 2003/0060979, filed on September 21, 2001; 
hereinafter Andrews ) in view of Brechner et al. {Pat. No. US 6,741,996, filed on 
April 18, 2001; hereinafter Brechner ), further in view of La Due (Pub. No. US 
2003/0133423, filed on November 15, 2002) and further in view of Tecu et al. 
{Pub. No. US 2004/0034655, filed on July 17, 2002; hereinafter Tecu). and 
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further in view of Ijuruoka {Pat. No. US 6,192,056, published on February 20, 
2001), and further in view of Lew et al. (Pub. No. US 2002/0031240, filed on 
December 6, 2000; liereinafter Lew ). 



Regarding claim 25, Tecu further discloses extracting metadata using a 
magnitude value {various intensity ranges 74 may also be used to designate a 

particular bit pattern of information. For example, a particular range of signal level 
strengttis may be used to identify a bit designation of "1 " while another range of 
signal level strengths may be used to identify a bit designation of "0", [001 7] of 
Tecu ). 

Andrews , as modified by Brechner . LaDue . Tecu . and Tsuruoka . does not 
disclose the first two limitations. 

However, Lew discloses: 

obtaining magnitude ratio values of each pair of said magnitude signal 

values {For each of the M selected coefficients, x, the embedder computes a 
ratio of the magnitude of a selected coefficient relative to the magnitude of its 
neighbors (108). In particular, it is a ratio of the magnitude of the selected 
coefficient to the average magnitude of the surrounding neighbors, [0013]); 

normalizing said magnitude ratio values {The embedding and detecting 
operations apply to other media types, including audio media signals. In addition, 
the frequency domain coefficients may be selected and adjusted to reference 
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values to detect other types of signal alteration, sucli as lossy compression, 
digital to analog and analog to digital conversion, downsampling and upsampling, 
etc, [0031]); and 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of Lew with the 
teachings of Andrews , as modified by Brechner . LaDue . Tecu . and Tsuruoka . for 
the purpose of detecting and analyzing alteration of a watermarked media signal 
that contains data information by examining signal peaks at selected frequency 
coefficients in the media signal ([0008] of Lew ). 

1 3. Claim 40 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Nakamura et al. {Pub. No. US 2004/0201740, filed on March 15, 2002; 
hereinafter Nakamura ) in view of Andrews et al. (Pub. No. US 2003/0060979, 
filed on September 21, 2001; hereinafter Andrews ), and further in view of 
Brechner et al. {Pat. No. US 6,741,996, filed on April 18, 2001; hereinafter 
Brechner ), and further in view of Kometani et al. (Pat. No. US 5,805,773, 
published on September 8, 1998; hereinafter Kometani ). 

Regarding claim 40, Nakamura . as modified by Andrews and Brechner . 
does not explicitly disclose the limitation of this instant claim. 

However, Kometani discloses deriving said membership grade value 
further comprises at least one out of averaging said membership function over 
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said first period of time, determining a maximum of said memberslnip function 

over said first period of time, and determining a minimum of said membersliip 
function over said first period of time {According to Formula (i), as shown in 
Figure 7, tlie membersliip functions (indicated by solid lines) of the conclusion 
part propositions M6, M7 and M8 of the rules R1, R2 and R3 are respectively 
weighted by the condition part membership grades 0.4, 0.9 and 0.3 into the 
curves of the fuzzy-rule membership grades (indicated by broken lines), [Column 
5, Lines 56-62]). 

It would have been obvious to a person with ordinary skills in the art at the 
time of the invention was made to incorporate the teachings of Lew with the 
teachings of Andrews , as modified by Brechner . LaDue . Tecu . and Tsuruoka . for 
the purpose of selecting one fuzzy rule group and at least one fuzzy set in 
conformity with a situation at that time and combines them and executing the 
reasoning for fact information ([Abstract] of Kometani ). 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
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action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1 .1 36(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 



Contact Information 

14. Any inquiry concerning this communication or earlier communications from 
the Examiner should be directed to Son T. Hoang whose telephone number is 
(571) 270-1752. The Examiner can normally be reached on Monday - Friday 
(7:00 AM -4:00 PM). 

If attempts to reach the Examiner by telephone are unsuccessful, the 
Examiner's supervisor, Neveen Abel-Jalil can be reached on (571) 272-4074. 
The fax phone number for the organization where this application or proceeding 
is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through Private 
PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
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free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 

/Son T Hoang/ 
Examiner, Art Unit 2165 
July 30, 2009 

/Neveen Abel-Jalil/ 

Supervisory Patent Examiner, Art Unit 2165 



